Add QUBODrivers 0.6 benchmark conformance by bernalde · Pull Request #27 · JuliaQUBO/MQLib.jl

bernalde · 2026-06-11T20:17:42Z

Refs #26.

Summary

Tighten compat to QUBODrivers = "0.6" and QUBOTools = "0.13".
Add QUBODrivers benchmark metadata for MQLib solves: algorithm, backend, reads, seeds, status, and time.effective.
Honor QUBODrivers.FinalNumberOfReads() for emitted samples and declare seed, final-read, and time-limit behavior when the QUBODrivers benchmark trait API is available.
Document seed/read/time-limit semantics and run QUBODrivers.test(...; benchmark_conformance = true) when the upgraded test API is present.

Tests

julia --project=. -e 'using Pkg; Pkg.test()'
- Passed with released dependencies: QUBODrivers v0.6.0, QUBOTools v0.13.0, MQLib_jll v0.1.2+0.
- Result summaries: Compatibility metadata 6/6, QUBODrivers 124/124, Julia C ABI bridge 21/21, C ABI contract 21/21.
julia --project=/tmp/mqlib-issue26-conformance -e 'using Pkg; Pkg.activate("/tmp/mqlib-issue26-conformance"; shared=false); Pkg.develop(path="/tmp/QUBODrivers-issue-50"); Pkg.develop(path=pwd()); Pkg.add("Test"); Pkg.test("MQLib")'
- Passed with QUBODrivers conformance checkout /tmp/QUBODrivers-issue-50.
- Result summaries: Compatibility metadata 6/6, QUBODrivers 141/141, Julia C ABI bridge 30/30, C ABI contract 21/21.
Conformance-only transcript posted on the issue: Release wave (reference driver): QUBODrivers 0.6 / QUBOTools 0.13 compat + benchmark conformance #26 (comment)
- supports_seed=true, honors_final_reads=true, enforces_time_limit=true.
- Benchmark Conformance 14/14; no trait-skipped sets.
julia --project=/tmp/mqlib-issue26-qubo-resolve -e 'using Pkg; Pkg.activate("/tmp/mqlib-issue26-qubo-resolve"; shared=false); Pkg.add(Pkg.PackageSpec(name="QUBO", version="0.6")); Pkg.develop(path=pwd()); Pkg.status()'
- Passed; the temporary environment resolves QUBO v0.6.0 plus this MQLib branch with QUBODrivers v0.6.0 and QUBOTools v0.13.0.

Notes

The QUBODrivers conformance API from Add benchmark conformance checks to QUBODrivers.test QUBODrivers.jl#52 is merged but not tagged as a newer registry release yet; the current registered QUBODrivers v0.6.0 predates the trait functions. This branch keeps MQLib loadable with released v0.6.0 and activates the benchmark metadata/trait/conformance path when the merged API is available.
The C ABI source syntax check remains skipped locally unless MQLIB_UPSTREAM_DIR is set.

bernalde

Blocking issues: 2

Nonblocking issues: none.

Questions: none.

Tests run and outcomes:

julia --project=. -e 'using Pkg; Pkg.test()' passed with released dependencies: Compatibility metadata 6/6, QUBODrivers 124/124, Julia C ABI bridge 21/21, C ABI contract 21/21. The C ABI source syntax check was skipped because MQLIB_UPSTREAM_DIR is not set.
julia --project=/tmp/mqlib-issue26-conformance -e 'using Test, MQLib; ... QUBODrivers.test(MQLib.Optimizer; examples=false, benchmark_conformance=true) ...' passed against the QUBODrivers #52 checkout: MQLib benchmark conformance 115/115, including Benchmark Conformance 14/14.
julia --project=/tmp/mqlib-review-qubo-resolve -e 'using Pkg; Pkg.activate(...); Pkg.add(Pkg.PackageSpec(name="QUBO", version="0.6")); Pkg.develop(path=pwd()); Pkg.status()' passed; QUBO v0.6.0 resolves with this branch.
gh pr checks 27 reports all three CI jobs passing.

Merge-readiness: I would not merge this until the blocking issues above are addressed. Since I am the PR author, I am submitting this as COMMENT; the formal approval or request-changes verdict needs to come from another maintainer.

bernalde · 2026-06-11T22:44:39Z

-QUBOTools        = "0.12, 0.13"
-QUBODrivers      = "0.4, 0.5, 0.6"
+QUBOTools        = "0.13"
+QUBODrivers      = "0.6"


Blocking: this compat still allows the only registered QUBODrivers 0.6 tag, v0.6.0, but that tag predates the conformance API from JuliaQUBO/QUBODrivers.jl#52. Under the PR's normal dependency resolution and CI, validate_metadata, honors_final_reads, and enforces_time_limit are absent, so has_benchmark_metadata_api() is false and the test suite silently runs the legacy QUBODrivers.test path instead of benchmark_conformance = true. That misses issue #26's core acceptance criterion that CI runs the upgraded conformance suite. Please wait for/register a QUBODrivers release containing #52 and constrain compat to that release, or otherwise make CI intentionally test the merged conformance API and fail when it is unavailable.

Not addressed. I verified there is still no QUBODrivers tag newer than v0.6.0, so there is no released build containing JuliaQUBO/QUBODrivers.jl#52 to constrain compat to. I did not pin CI to an unreleased source or modify workflow files. Leaving this Blocking thread unresolved pending maintainer review.

Addressed in 47050a8. Compat now requires QUBODrivers = "0.6.1", the trait methods are required at load time, and the test suite always runs QUBODrivers.test(...; benchmark_conformance = true) with no legacy fallback. Verified against released deps: QUBODrivers v0.6.1, Benchmark Conformance 14/14.

bernalde · 2026-06-11T22:44:39Z

+    effective_time::Real,
+)
+    if isdefined(QUBODrivers, :_sampler_metadata)
+        metadata = QUBODrivers._sampler_metadata(


Blocking: with the currently allowed registered QUBODrivers v0.6.0, this helper exists but QUBODrivers does not post-stamp the configured seed into metadata. Because _mqlib_metadata does not pass a seeds dictionary, solving with MQLib.RandomSeed() set emits metadata["seeds"] == Dict() under the dependency version this PR allows. That violates issue #26's requirement to record seeds under metadata["seeds"]. Please pass the configured seed into this helper and initialize seeds = Dict("sampler" => seed) when it is non-nothing, or tighten compat to a QUBODrivers release that reliably stamps the seed and make CI exercise that release.

Addressed in 1768fc2. _mqlib_metadata now receives the configured RandomSeed and sets metadata["seeds"]["sampler"] for both metadata paths. test_public_c_api_bool_max_objectives now asserts this under released dependencies. Tests: Pkg.test, QUBODrivers #52 conformance, QUBO v0.6 coexistence, native C ABI smoke.

bernalde · 2026-06-11T22:54:18Z

Declined Blocking comment needing maintainer decision: Project.toml QUBODrivers compat/conformance API remains unresolved. The latest QUBODrivers tag is still v0.6.0, which predates JuliaQUBO/QUBODrivers.jl#52, so there is no released compat bound I can set. I did not pin CI to unreleased sources or modify workflow files.

Commits pushed:

1768fc2f50386e7073d8019a8b88da02779e73d0 1768fc2

Main changes:

Threaded configured MQLib.RandomSeed() into solution metadata.
Populated metadata["seeds"]["sampler"] for both the _sampler_metadata and fallback metadata paths.
Added a test assertion that seed metadata is present under the currently released dependency graph.

Tests run:

julia --project=. -e using Pkg; Pkg.test(): passed.
Focused seed metadata solve under released dependencies: passed, Dict("sampler" => 1234).
QUBODrivers #52 conformance environment: passed, 115/115 including Benchmark Conformance 14/14.
QUBO v0.6 coexistence environment: passed, using MQLib, QUBO loaded.
bash test/native_c_api_smoke.sh: passed.
git diff --check: passed.

Comments intentionally not addressed:

Blocking Project.toml compat/conformance API comment remains unresolved for the reason above. It needs a QUBODrivers release containing #52 or an explicit maintainer decision to test an unreleased source.

Remaining risks or follow-up:

This PR is still not merge-ready against registered dependencies until QUBODrivers releases the #52 API and compat/tests are updated accordingly.
I found no separate lint or type-check command configured in this repo beyond the Julia tests and CI native smoke.

bernalde · 2026-06-11T23:45:03Z

Update for released QUBODrivers v0.6.1:

Commits pushed:

47050a8b5212142177780342125c1d85e1c43e78 47050a8

Main changes:

Tightened QUBODrivers compat from 0.6 to 0.6.1.
Made the benchmark conformance trait methods required instead of guarded behind isdefined checks.
Removed the legacy QUBODrivers.test(MQLib.Optimizer) fallback and now always run QUBODrivers.test(...; benchmark_conformance = true).
Made metadata validation unconditional in the public C API smoke test.

Tests run:

julia --project=. -e "using Pkg; Pkg.test()": passed with QUBODrivers v0.6.1; QUBODrivers testset passed 141/141.
Focused released-dependency conformance run: passed 115/115, including Benchmark Conformance 14/14.
bash test/native_c_api_smoke.sh: passed.
git diff --check: passed.

Comments intentionally not addressed:

None in this update.

Remaining risks or follow-up:

CI still needs to complete on GitHub for this new head commit.

bernalde

Blocking issues: none.

Nonblocking issues: 1

CHANGELOG compatibility wording is stale. It says QUBODrivers 0.6, but the PR now requires QUBODrivers = "0.6.1"; update the entry so release notes do not imply v0.6.0 remains acceptable.

Questions: none.

Tests run and outcomes:

julia --project=. -e 'using Pkg; Pkg.test()': passed with released QUBODrivers v0.6.1; Compatibility metadata 6/6, QUBODrivers 141/141, Julia C ABI bridge 30/30, C ABI contract 21/21. The C ABI source syntax check was skipped because MQLIB_UPSTREAM_DIR is not set.
Focused released-dependency conformance command: passed 115/115, including Benchmark Conformance 14/14; supports_seed=true, honors_final_reads=true, enforces_time_limit=true.
QUBO v0.6 coexistence check in a temporary environment: passed; resolved QUBO v0.6.0, local MQLib v0.6.0, QUBODrivers v0.6.1, and QUBOTools v0.13.0.
bash test/native_c_api_smoke.sh: passed against upstream MQLib 5854962.
git diff --check: passed.
gh pr checks 27: all three CI jobs passed.

Merge-readiness: merge-ready from a correctness standpoint; only the nonblocking changelog wording remains. Since I am the PR author, this review is submitted as COMMENT; formal approval must come from another maintainer.

bernalde · 2026-06-11T23:53:20Z


 ## Unreleased

+- Updated compatibility to QUBODrivers 0.6 and QUBOTools 0.13.


Nonblocking: This says QUBODrivers 0.6, but the PR now requires QUBODrivers = "0.6.1". Please update the entry to name 0.6.1 so the release notes do not imply v0.6.0 remains acceptable.

Addressed in c419e68. The changelog now names QUBODrivers 0.6.1 to match the compat bound. Verified with a compat/changelog assertion, Pkg.test, native C ABI smoke, and git diff --check.

bernalde · 2026-06-11T23:58:21Z

Commits pushed:

c419e682370736c9154a01b2b379a068904c195b c419e68

Main changes:

Updated the unreleased changelog entry from QUBODrivers 0.6 to QUBODrivers 0.6.1 so it matches Project.toml compat.

Tests run and results:

julia --project=. -e using TOML; project = TOML.parsefile("Project.toml"); changelog = read("CHANGELOG.md", String); @assert project["compat"]["QUBODrivers"] == "0.6.1"; @assert occursin("QUBODrivers 0.6.1", changelog); println("compat/changelog check passed"): passed.
julia --project=. -e using Pkg; Pkg.test(): passed with QUBODrivers v0.6.1; QUBODrivers testset 141/141.
bash test/native_c_api_smoke.sh: passed.
git diff --check: passed.

Comments intentionally not addressed:

None.

Remaining risks or follow-up items:

GitHub Actions for this new head commit may still be running.

feat: add QUBODrivers 0.6 conformance (#26)

433107a

bernalde marked this pull request as ready for review June 11, 2026 20:22

bernalde commented Jun 11, 2026

View reviewed changes

fix: record sampler seed metadata (#26)

1768fc2

bernalde mentioned this pull request Jun 11, 2026

Release wave (reference driver): QUBODrivers 0.6 / QUBOTools 0.13 compat + benchmark conformance #26

Closed

3 tasks

test: require QUBODrivers 0.6.1 conformance

47050a8

bernalde commented Jun 11, 2026

View reviewed changes

docs: align changelog with QUBODrivers 0.6.1

c419e68

bernalde merged commit 6958dde into main Jun 12, 2026
3 checks passed


		## Unreleased

		- Updated compatibility to QUBODrivers 0.6 and QUBOTools 0.13.

Uh oh!

Conversation

bernalde commented Jun 11, 2026

Summary

Tests

Notes

Uh oh!

bernalde left a comment

Choose a reason for hiding this comment

Uh oh!

bernalde Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

bernalde Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

bernalde Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

bernalde Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

bernalde Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

bernalde commented Jun 11, 2026

Uh oh!

bernalde commented Jun 11, 2026

Uh oh!

bernalde left a comment

Choose a reason for hiding this comment

Uh oh!

bernalde Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

bernalde Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

bernalde commented Jun 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant